Skip to content

feat(sec): the codebud-sec command, evidence log on every step (step 5, PR 7) - #23

Open
mkhellat wants to merge 4 commits into
mainfrom
feat/codebud-sec
Open

feat(sec): the codebud-sec command, evidence log on every step (step 5, PR 7)#23
mkhellat wants to merge 4 commits into
mainfrom
feat/codebud-sec

Conversation

@mkhellat

@mkhellat mkhellat commented Nov 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Phase 10, step 5 plan, PR 7 (D14a): the codebud-sec command, the evidence log wired to every step, and a manual chapter, "Security Work".

Commits

  1. c61aa3c Step recorders (core). With add_step_recorder(), the executor tells a recorder about every step: before it runs (start), after it ends (done, also when the tool crashed or returned a malformed result), and when it is refused (refused). A step whose start can't be recorded doesn't run, so nothing runs unrecorded.
  2. 02c4a42 The codebud-sec command.
    • codebud-sec chat|run|plan --engagement FILE loads the engagement and opens its evidence log: created encrypted beside the file, or reopened with its chain checked. Then it activates the scope guard, the network fence and the evidence recorder, and hands over to codebud with security-pentest, unless --profile names another. Other options pass through.
    • When codebud ends, everything is taken down and the session's exit status is recorded.
    • Output over 64 KiB is kept as its SHA-256, size and beginning.
    • evidence add records files with their SHA-256 and a note. verify [--files] checks the chain and prints the head hash; show lists the entries; close ends the log.
    • pin is codebud integrity pin.
    • CORE_API += codebud.cli.main, a deliberate widening.
    • Both make install (a second launcher) and pip (project.scripts) install the command.
  3. 165b098 Its own hint for scope refusals. They used to get the allow-list advice, and no allow-list can widen an engagement's scope.

Verification

  • make check: 1796 passed. make distcheck installs and uninstalls both commands. The info and HTML manuals build without warnings; ruff is clean.
  • Live, with Ollama and scratch config dirs:
    • codebud-sec pin pinned 2 models and 71 binaries, pasta, nft and sh among them.
    • A planned nmap step without --unprivileged was refused at plan time, with the fence's reason.
    • With --unprivileged, codebud-sec run ran it behind the fence and it found the router's ports. The log held the step's start and done records, and evidence verify reported the chain intact.

@mkhellat
feat(safety): step recorders, told about every step the executor runs
c61aa3c 
Phase 10, step 5 plan, PR 7.  The security layer's evidence log must
hold every step of an engagement.  A layer registers a recorder with
add_step_recorder(); the executor tells it about each step before it
runs ("start"), after it ends ("done", with its result, also when the
tool crashed or returned a malformed result) and when the safety engine
refuses it ("refused").  A step whose start cannot be recorded does not
run, so nothing runs unrecorded; a failure to record the end is added
to the step's error output.

* src/codebud/safety.py (_STEP_RECORDERS, add_step_recorder)
(remove_step_recorder, record_step): New.
* src/codebud/executor.py (Executor.execute_plan): Record each step.
* tests/test_step_checks.py: Test them.
* docs/chapters/safety-and-sandbox.texi (The Safety Engine): Document
them.
@mkhellat
feat(sec): the codebud-sec command
02c4a42 
Phase 10, step 5 plan, PR 7 (D14a).  codebud-sec chat|run|plan
--engagement FILE runs codebud inside an engagement: it loads the
engagement, opens its evidence log (created encrypted beside the file,
<name>.evidence.jsonl, or the one --evidence names; reopened with its
chain checked for later sessions), activates the scope guard and
network fence, registers the evidence recorder, and hands over to
codebud with a security profile -- security-pentest unless --profile
names another; other options pass through.  When codebud ends,
everything is taken down and the session's end and exit status are
recorded.

Every step goes into the log before it runs and after it ends, refused
steps as warnings; text over 64 KiB is kept as its SHA-256, size and
head.  codebud-sec evidence add records files with their SHA-256 and a
note; verify checks the chain (--files: the recorded files too) and
prints the head hash to keep elsewhere; show lists the entries; close
ends the log.  codebud-sec pin is codebud integrity pin.

Verified live with Ollama: a planned nmap step without --unprivileged
was refused at plan time; with it, codebud-sec run ran it behind the
fence, and the log held both records and verified.

The layer now calls codebud's own main: CORE_API += codebud.cli.main.
make install and pip install both provide the command.

* src/codebud_sec/cli.py, src/codebud_sec/__main__.py: New files.
* pyproject.toml (project.scripts): Add codebud-sec.
* Makefile.am (bin_SCRIPTS): Build codebud-sec from the launcher.
* src/launcher.in: Name the module and command.
* src/Makefile.am: Install the new modules.
* .gitignore: Ignore the built launcher.
* tests/test_layers.py (CORE_API): Add codebud.cli.main.
* tests/test_codebud_sec.py: New file.
* docs/chapters/security.texi: New chapter, "Security Work".
* docs/codebud.texi, docs/Makefile.am: Include it.
* docs/chapters/architecture.texi, docs/chapters/development.texi:
Mention them.
* README.md, NEWS: Describe codebud-sec.
@mkhellat
feat(cli): a scope refusal gets its own hint
165b098 
A step the engagement's scope refused was explained as a safety policy
refusal, whose hint is to add the command to an allow-list -- which
cannot widen an engagement's scope.  It now says the step reaches
outside the engagement or cannot be read, and to rephrase it so each
step names only in-scope targets and meets the reason given.

* src/codebud/cli/errors.py (_RULES): Add "engagement scope" before
"safety policy".
* tests/test_codebud_sec.py (test_a_scope_refusal_gets_its_own_hint):
New test.
@mkhellat

mkhellat commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

Review summary: changes requested

One full sweep over 165b098: conditions → scenarios → each checked against the code → De Morgan.

Conditions.

  • A session runs ⇔ the engagement loads ∧ a key exists ∧ the log opens (created, or its chain verifies).
  • While it runs: provided("engagement") ∧ the fence is registered ∧ the recorder is registered.
  • A step runs ⇒ its start record was written.

The invariants behind them:

  1. The log can only be appended to through codebud-sec.
  2. Every session works under the scope its log names.
  3. Codebud's outcome reaches the operator.

Correct, verified (1796 tests, plus live runs with Ollama):

  • The recorder covers start, done, refused, crashes and malformed results, and a step whose start can't be recorded doesn't run.
  • Guard, fence and recorder are taken down in finally, including on SystemExit.
  • parse_known_args passes codebud's own options through.
  • pin is integrity pin.
  • The layer imports only CORE_API (+ codebud.cli.main, deliberate).
  • distcheck installs and uninstalls both commands.

Defects (each reproduced on 165b098):

  1. The default log can be rewritten by a step (invariant 1). <engagement>.evidence.jsonl beside the engagement file usually lies in the project, which steps may write: confine.check_write("lab.evidence.jsonl") accepts it, and any command step can write there too. A step could truncate the log or replace it consistently. Fix: by default the log goes in codebud's data directory, which steps never see (bwrap hides it, and the file tools are confined to the project). An --evidence path inside the project is refused.
  2. The engagement file isn't checked against the log (invariant 2). With in_scope widened to 0.0.0.0/0 between sessions (a step could do that too, if the file is in the project), the next session opened on the same log with the new scope. Fix: on reopening, the engagement file's SHA-256 must match the one in the log's open entry; a changed file is refused, with the advice to start a new log.
  3. A message codebud exits with is lost (invariant 3). sys.exit("…") from codebud became status 1 with no text. Fix: print the message, as Python would.

Verdict: changes requested. The fixes come as new commits, then a re-review.

@mkhellat
fix(sec): keep the evidence log out of the steps' reach, tied to its …
a3eb006 
…engagement

Review of #23.  Three defects, each reproduced:

The default log lay beside the engagement file, usually in the
project, which steps may write (file_write accepted it; any command
step could write it too): a step could truncate it or replace it
consistently.  It now lives in codebud's data directory,
evidence/<engagement>-<id>.jsonl, which steps never see; an --evidence
path inside the project is refused.

The engagement file was reloaded at each session without comparing it
to the SHA-256 the log's open entry records: a scope widened to
0.0.0.0/0 between sessions opened on the same log.  Reopening a log now
unlocks the key, checks the whole chain, and refuses an engagement
file other than the one the log was opened with.

A message codebud exited with (sys.exit("...")) became status 1 with
no text; it is printed now, as Python would.

CORE_API grows by codebud.paths (the data directory) and
codebud.confine (the project root), deliberately.

* src/codebud_sec/cli.py (evidence_path): The data directory by
default; refuse a path inside the project.
(_open_log): Unlock, check the chain and the engagement file.
(_session): Show codebud's exit message.
(_log_for): Report a refused path.
* tests/test_layers.py (CORE_API): Add codebud.paths, codebud.confine.
* tests/test_codebud_sec.py: Follow the new place; test the fixes.
* docs/chapters/security.texi (Running codebud-sec), NEWS: Say where the
log is kept and that a changed scope needs a new log.
@mkhellat

mkhellat commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

Re-review summary: fixed, ready to merge once CI passes

Second sweep, over a3eb006.

Finding Fix Verified
1. A step could rewrite the default log (in the project) The log now goes in codebud's data directory (evidence/<engagement>-<id>.jsonl), which bwrap hides and the file tools can't reach; an --evidence path inside the project is refused confine.check_write refuses the new path; case.jsonl, logs/case.jsonl and . are refused; live run
2. The engagement file wasn't checked against the log Reopening unlocks the key, checks the whole chain by its contents, and compares the engagement file's SHA-256 with the open entry; a change is refused, with the advice to start a new log widened to 0.0.0.0/0 → refused, for sessions and evidence add; live run
3. codebud's exit message was lost Printed, as Python would; status 1 test and live run

This sweep:

  • Reopening now asks for the passphrase once per session. That is the cost of checks 2 and 1's full chain verification, and the chapter says so.
  • CORE_API grows by codebud.paths and codebud.confine, each with its reason in tests/test_layers.py.

1802 tests pass, make distcheck passes, make info builds without warnings, and ruff is clean. Verdict: correct. I'll merge with a merge commit once CI passes on all jobs, macOS included.

@mkhellat mkhellat self-assigned this Nov 12, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@mkhellat mkhellat

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@mkhellat